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ANALYSIS OF CHEMICAL DATA FROM IMAGES 



CROSS REFERENCE TO RELATED APPLICATIONS 
[0001] This application is a continuation of Application No. 09/415,772, filed October 8, 1999, 
which is a continuation of and claims priority to International Application PCT/US99/07358, 
with an international filing date of April 1, 1999, now abandoned, which is a continuation-in-part 
of Application No. 09/227,558, filed January 8, 1999, which is a continuation-in-part of 
Application No. 08/898,715, filed July 22, 1997, which claims the benefit of Provisional 
Application Nos. 60/050,949, filed June 13, 1997; 60/028,106, filed October 9, 1996; 
60/029,255, filed October 25, 1996; 60/035,366, filed January 10, 1997; 60/048,987, filed June 
9, 1997; 60/028,105, filed October 9, 1996; and 60/035,202, filed January 10, 1997. Each of the 
foregoing applications is incorporated herein by reference in its entirety and is the basis of a 
claim for priority under 35 U.S.C. § 1 19 or 120. 

BACKGROUND OF THE INVENTION 
[0002] The present invention relates to methods and computer programs for rapidly screening 
and characterizing materials by the analysis of data acquired from images. 
[0003] In combinatorial chemistry, a large number of candidate materials are created from a 
relatively small set of precursors and subsequently evaluated for suitability for a particular 
application. As currently practiced, combinatorial chemistry permits scientists to systematically 
explore the influence of structural variations in candidates by dramatically accelerating the rates 
at which they are created and evaluated. Compared to traditional discovery methods, 
combinatorial methods sharply reduce the costs associated with preparing and screening each 
candidate. 

[0004] Combinatorial chemistry has revolutionized the process of drug discovery. One can 
view drug discovery as a two-step process: acquiring candidate compounds through laboratory 
synthesis or through natural products collection, followed by evaluation or screening for 
efficacy. Pharmaceutical researchers have long used high-throughput screening (HTS) protocols 
to rapidly evaluate the therapeutic value of natural products and libraries of compounds 
synthesized and cataloged over many years. However, compared to HTS protocols, chemical 
synthesis has historically been a slow, arduous process. With the advent of combinatorial 
methods, scientists can now create large libraries of organic molecules at a pace on par with HTS 
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protocols. 

[0005] Recently, combinatorial approaches have been used for discovery programs unrelated to 
drugs. For example, some researchers have recognized that combinatorial strategies also offer 
promise for the discovery of inorganic compounds such as high-temperature superconductors, 
magneto resistive materials, luminescent materials, and catalytic materials. See, for example, 
U.S. Patent No. 5,776,359 and International Patent Publication No. WO 98/03251, which are 
both incorporated herein by reference. 

SUMMARY 

[0006] The invention provides computer programs and computer-implemented methods for 
extracting and analyzing combinatorial chemical data from images. 

[0007] In general, in one aspect, the invention features a computer program for evaluating a 
combinatorial library including a plurality of members. The program includes instructions to 
receive a stream of data including a series of images of the combinatorial library; to identify a 
plurality of regions of interest, each region corresponding to a location in each of the series of 
images and to a location in the combinatorial library; to determine a series of values for one or 
more regions of interest, the series of values for each of the one or more regions including a 
value for each of the images; and to calculate from each series of values for the one or more 
regions a figure of merit for the library member at the corresponding library location. 
[0008] Implementations of the invention can include one or more of the following advantageous 
features. Each region of interest corresponds to a plurality of pixels in each image and the series 
of values for a region of interest comprises an average value for each of the corresponding 
plurality of pixels in each of the images. The stream of data comprises a series of images 
generated at a frequency of greater than about 1 frame per second. The stream of data comprises 
a series of images generated at a frequency of greater than about 6 frames per second. The 
stream of data comprises a series of images generated at a frequency of greater than about 12 
frames per second. The stream of data comprises a series of images generated at a frequency of 
greater than about 20 frames per second. The program further includes instructions to display a 
graphical representation of the figures of merit. The graphical representation includes a 
histogram. The series of images is captured from a beginning of a combinatorial experiment to 
an end of the combinatorial experiment and the plurality of regions of interest may be identified 
after the series of images has been captured or after one or more images in the series of images 
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has been captured. The series of images includes a series of infrared images. The figure of merit 
comprises an emittance change for a library member. The figure of merit comprises a phase 
transition point for a library member. The figure of merit comprises a thermoelectric figure of 
merit for a library member. The instructions to identify a plurality of regions of interest 
comprise instructions to receive a first user input identifying one or more regions of interest, 
each region having a user-defined shape. The program further includes instructions to receive a 
second user input defining a computation for determining a series of values for each of the one or 
more regions of interest. The instructions to determine a series of values for each of the one or 
more regions of interest include instructions to compute each value in each series in accordance 
with the second user input. The computation is an average for a group of pixels corresponding to 
a region of interest. 

[0009] In general, in another aspect, the invention features a computer program on a computer- 
readable medium for evaluating a combinatorial library including a plurality of members. The 
program includes instructions to receive a stream of data including a series of images of the 
combinatorial library; to receive a first user input identifying one or more regions of interest, 
each region having a user-defined shape and each region corresponding to a location in each of 
the series of images and to a location in the combinatorial library; to receive a second user input 
defining a computation for determining a series of values for each of the one or more regions of 
interest, each series of values including a value for each of the images; and to determine a series 
of values for each of the one or more regions of interest, each value in each series being 
computed in accordance with the second user input. 

[0010] Implementations of the invention can include one or more of the following advantageous 
features. The program further includes instructions to calculate from each series of values for the 
one or more regions a figure of merit for the library member at the corresponding library 
location. 

[001 1] In general, in another aspect, the invention features a computer program for evaluating a 
combinatorial chemical experiment. The program includes instructions to receive a series of 
images of a combinatorial library, the library including a plurality of members, the series of 
images captured from a beginning of the experiment to an end of the experiment; to identify, 
after the series of images has been captured, a plurality of regions of interest, each region 
corresponding to a location in each of the series of images and to a location in the combinatorial 
library; and to determine a series of values for one or more regions of interest, the series of 
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values for each of the one or more regions including a value for each of the images. 
[0012] In general, in another aspect, the invention features a method for evaluating a 
combinatorial library including a plurality of members. The method includes receiving a stream 
of data including a series of images of the combinatorial library; identifying a plurality of regions 
of interest, each region corresponding to a location in each of the series of images and to a 
location in the combinatorial library; determining a series of values for one or more regions of 
interest, the series of values for each of the one or more regions including a value for each of the 
images; and calculating from each series of values for the one or more regions a figure of merit 
for the library member at the corresponding library location. 

[0013] In general, in another aspect, the invention features a computer-implemented method for 
evaluating a combinatorial chemical experiment. The method includes capturing a series of 
images of a combinatorial library, the library including a plurality of members, the series of 
images being captured from a beginning of the experiment to an end of the experiment and 
storing the series of images in memory; after the series of images has been captured, identifying 
a plurality of regions of interest, each region corresponding to a location in each of the series of 
images and to a location in the combinatorial library; and determining a series of values for one 
or more regions of interest, the series of values for each of the one or more regions including a 
value for each of the images. 

[0014] In general, in other aspect, the invention features a system for evaluating a combinatorial 
library including a plurality of members. The system includes means for receiving a stream of 
data comprising a series of images of the combinatorial library; means for identifying a plurality 
of regions of interest, each region corresponding to a location in each of the series of images and 
to a location in the combinatorial library; means for determining a series of values for one or 
more regions of interest, the series of values for each of the one or more regions comprising a 
value for each of the images; and means for calculating from each series of values for the one or 
more regions a figure of merit for the library member at the corresponding library location. 
[0015] In general, in other aspect, the invention features a system for evaluating a combinatorial 
library, the library including a plurality of members. The system includes means for receiving a 
stream of data comprising a series of images of the combinatorial library; means for receiving a 
first user input identifying one or more regions of interest, each region having a user-defined 
shape and each region corresponding to a location in each of the series of images and to a 
location in the combinatorial library; means for receiving a second user input defining a 
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computation for determining a series of values for each of the one or more regions of interest, 
each series of values comprising a value for each of the images; and means for determining a 
series of values for each of the one or more regions of interest, each value in each series being 
computed in accordance with the second user input. 

[0016] In general, in other aspect, the invention features a system for evaluating a combinatorial 
chemical experiment. The system includes means for receiving a series of images of a 
combinatorial library, the library including a plurality of members, the series of images captured 
from a beginning of the experiment to an end of the experiment; means for identifying a plurality 
of regions of interest after the series of images has been captured, each region corresponding to a 
location in each of the series of images and to a location in the combinatorial library; and means 
for determining a series of values for one or more regions of interest, the series of values for each 
of the one or more regions comprising a value for each of the images. 

[001 7] Advantages that can be seen in implementations of the invention include one or more of 
the following. The use of a video image sequence allows the automated extraction of data from 
one or more regions in every frame of the image set. This results in a time-resolved profile of 
the reaction or transformation being observed, rather than merely snapshots based on a limited 
number of selected images or image frames. Regions of interest in each image can be defined 
after the image sequence is captured. Calculations and corrections can be applied automatically 
to every region in every frame of the image set. One or more figures of merit can be extracted 
from some or all of the data set for each region and can be graphically displayed. 
[0018] The details of one or more embodiments of the invention are set forth in the 
accompanying drawings and the description below. Other features and advantages of the 
invention will become apparent from the description, the drawings, and the claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0019] The file of this patent contains at least one drawing executed in color. Copies of this 
patent with color drawing(s) will be provided by the Patent and Trademark Office upon request 
and payment of the necessary fee. 

[0020] FIG. 1 is a block diagram of a data processing system showing an implementation of the 
invention. 

[0021] FIG. 2 is a flow diagram of a method of reducing a data stream. 
[0022] FIG. 3 is a flow diagram illustrating the method of FIG. 2 in more detail. 
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[0023] FIG. 4 is a flow diagram of a method of generating a mask. 
[0024] FIG. 5A. is a diagram of a combinatorial library. 
[0025] FIG. 5B is an illustration of a dialog window for creating a mask. 
[0026] FIG. 5C is an illustration of selecting the corners of a mask. 

[0027] FIG. 6 is a flow diagram of a method of analyzing reduced data values and displaying 
calculated results. 

[0028] FIG. 7 is a flow diagram of a method of selecting display parameters and a display 
format. 

[0029] FIGS. 8A-8C are illustrations of formats for displaying experimental results. 
[0030] FIG. 9 is a window displaying images in data stream. 
[0031] FIG. 10 is a schematic diagram of a thermoelectric device. 

[0032] FIG. 1 1 is a schematic diagram of a thermoelectric device including a combinatorial 
array of thermoelectric materials. 

[0033] FIGS. 12A-12B are illustrations of display formats for a combinatorial thermoelectric 
materials experiment. 

[0034] FIG. 13 is a schematic diagram of a combinatorial experiment to monitor heats of 
reaction. 

[0035] FIG. 14 is a diagram illustrating the combinatorial library of FIG. 13 in more detail. 
[0036] FIG. 15 is a diagram of a combinatorial experiment to determine phase transition points 
of a library of materials. 

[0037] FIG. 16 is a schematic diagram of a computer platform suitable for implementing the 
data processing system of the invention. 

[0038] Like reference numbers and designations in the various drawings indicate like elements. 

DETAILED DESCRIPTION 
[0039] In the present invention, a camera observes an experiment. The resulting image data is 
analyzed by a data processing system implementing methods of the present invention, as will be 
described. 

[0040] Referring to FIG. 1, camera 100 observes experiment 110 and records one or more 
images representing experiment 110. In the exemplary system that will be described, camera 
100 is a commercially available high performance infrared camera, such as the SE-IR 
CamIRa™, available from SE-IR Corporation of Goleta, California. Such cameras can be used 
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to measure thermal properties such as radiant or photon emittance, exitance, thermal flux and the 
like. Commercially available software, such as SE-IR Corporation's CamIRa™ software 
package, can be used to drive the camera's operations and to generate the images. Experiment 
1 10 can be any experiment suitable for monitoring with camera 100. It can involve a single 
reaction vessel containing reagents or a single material deposited on a substrate within the field 
of view of camera 100. Alternatively, it can include a combinatorial array or "library" of distinct 
"members" (separate vessels or deposited materials), as will be discussed in more detail below. 
[0041] Image data 120 generated by camera 100 is stored in data store 130 as an array of picture 
elements or "pixels," each of which is represented by a specific intensity or pixel value. Data 
processing system 140 retrieves image data 120 from data store 130; alternatively, data 
processing system 140 can receive image data 120 directly from camera 100 without intervening 
storage in data store 130. After retrieving the image data, data processing system 140 processes 
the image data in data reduction module 150 and data analysis module 160 either automatically 
or under user control. A user can interact with system 140 through user interface module 170. 
Data processing system 140 displays results through display or printing devices 190, 196. 
[0042] As shown in FIG. 2, data reduction module 150 begins by getting a stream of data 
comprising a series of images from data store 130 or camera 100 (step 200). After getting the 
first image in the stream (step 210), data reduction module 150 reduces the data from that image, 
as will be described in further detail below (step 220). If additional images remain to be 
reduced (step 230), data reduction module 150 gets the next image (step 240) and reduces that 
image data (step 220). Data reduction module 150 provides the reduced data values to data 
analysis module 160 (step 250). Optionally, data processing system 140 stores the reduced data 
in data store 130. 

[0043] FIG. 3 describes data reduction step 220 in more detail. After receiving an image (step 
210), data reduction module 150 obtains a user-defined mask identifying one or more regions of 
interest in the image (step 300). These regions may correspond, for example, to reaction vessels 
or materials that make up the elements of a combinatorial library used in experiment 1 10. Data 
reduction module 150 applies the mask to the image (step 310), and extracts a value or values for 
each region of interest (step 320). Data reduction module 150 uses these values to calculate a 
reduced data value for each region of interest in the image (step 330). This reduced data value 
can be an average of intensity values recorded by camera 100 for points within a region of 
interest. It can also be an average change in intensity calculated by subtracting a reference point 
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value, for example, a value measured for the region at an earlier time (e.g., in a previous image), 
a value measured for background noise or some other value (such as reflectance or base line). In 
other embodiments, the reduced data value can be calculated using other known statistical 
functions. Optionally, data reduction module 150 allows the user to select a desired function, for 
example by choosing from a menu of possible functions. Data reduction module 150 produces a 
list of reduced data values for the array of regions of interest, which is sent to data analysis 
module 160 (step 340). The list of reduced data values may also be stored for future use. Data 
reduction module 150 then determines whether additional images remain to be reduced (step 
230). 

[0044] The process of creating a mask is described in FIG. 4. Through user interface 170, the 
user identifies one or more regions of interest within an image (step 400). As discussed above, 
these regions will generally correspond to specific reaction vessels or materials that make up the 
elements of a combinatorial library monitored by camera 100. Optionally, the user can also 
specify one or more reference regions to which experimental data can be compared (step 410). 
Data processing system 140 saves the resulting mask in data store 130 for subsequent use during 
experiment 1 10, as well as for use during later experiments sharing a similar substrate geometry 
(step 420), 

[0045] FIGS. 5A-5C detail the creation of a mask in one embodiment for an experiment 
involving a combinatorial library 500 of members 510 (FIG. 5 A). User interface 170 displays 
"Create Mask" dialog window 520 (FIG. 5B). The user can select a grid type 530, as well as the 
number of horizontal and vertical members in the grid, 540 and 550, respectively. The user can 
also select a member type 560, according to the nature of experiment 110. In another 
embodiment, data reducing module 150 allows the user to define the shape of mask regions to 
correspond to any shape of library members 510, for example, through a user interface similar to 
those found in commercially available drawing and painting software known to those skilled in 
the art. 

[0046] While FIG. 5B shows a "Create Mask" dialog for thermoelectric materials, the nature of 
the experiment is not a limiting feature for this invention as discussed below. The user can select 
a particular member length or width 570. After selecting the "Create" button, the user 
graphically specifies three corners 580 of the grid, clicks on the image and connecting lines are 
drawn (FIG. 5C). The elements of the grid can be moved as groups or individually to 
accommodate spatial irregularities in the physical substrate. Data reduction module 150 uses this 
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information to create the mask. 

[0047] As shown in FIG. 6, data analysis module 160 gets the list of reduced data values for 
each image from data reduction module 150 or data store 130 (step 600). Data analysis module 
160 uses the values to calculate experimental results, including one or more figures of merit for 
each region of interest (step 610), as will be described in more detail below. The user can save 
the calculated results (steps 620 and 630), and can view the displayed results in a number of 
graphical formats (steps 620 and 640). 

[0048] The user's interaction with data analysis module 160 (steps 620 through 640) is 
described in more detail in reference to FIG. 7. Through user interface 170, the user can select a 
variety of display options (step 700). These include a number of display formats (step 710) 
illustrated in FIGS. 8A-8C, such as an array of "thumbnail" graphs 800 plotting experimental 
data (for example, intensity versus time) for each region of interest a graph of experimental data 
for any selected region of interest (a "zoom-in" graph) 850, or a three- dimensional histogram 
880 displaying a calculated value for each region of interest. Data analysis module retrieves 
display parameters and control information (step 720) and gets the results calculated at step 610 
(step 730). Data analysis module 160 displays the results in the selected display format (step 
740). User selection step 700 also includes the option to choose display parameters (step 750). 
These include parameters such as histogram scale, range and colors, or time and value ranges for 
thumbnail or zoom-in graphs. By setting these parameters, the user can view experimental 
results over a specified time period, with the value at any specified time subtracted to show 
changes in temperature, or with a reference value (as described above) subtracted to remove 
background (or systematic) changes. 

[0049] As shown in FIG. 9, the user can also view the original image or images recorded by 
camera 100. Window 900 displays an infrared image 910 of experiment 110, such as microti ter 
plate 920. Depending on the user-selected display parameters, different colors in infrared image 
910 depict the range of heat emitted by objects within the field of camera 1 00. The user can 
navigate through the stream of images using buttons 930a-e provided in pane 940. Buttons 930d 
and 930b navigate forward or backward, respectively, by a single frame. Buttons 930e and 930a 
provide "fast forward" and "rewind" functions, respectively. In one embodiment, in "fast 
forward" mode data analysis module 160 and/or user interface module 170 determines the 
number of frames remaining in the stream of images and displays selected frames until it reaches 
the end of the stream. Similarly, in "rewind" mode, the number of preceding frames is 
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determined and selected frames are displayed in reverse order until the beginning of the image 
stream is reached. Thus, for example, fast forward mode can display every tenth frame in a 
stream including 1000 frames, for a total of 100 frames displayed. Pane 940 also displays the 
frame number of the image displayed. 

[0050] Data processing system 140 can be used to analyze material or chemical systems that 
lend themselves to characterization by IR thermography. Using IR thermography, the emittance 
(or exitance) of an entire library of materials can be monitored with an infrared camera and the 
measured values can be used to derive a variety of thermodynamic properties associated with the 
materials. Commercial position sensitive systems such as infrared focal plane arrays can have a 
high sensitivity over a wide temperature range. Commercial infrared cameras incorporating such 
systems can acquire data at speeds up to 100 or more frames per second. In combination with 
such a system, data processing system 140 is particularly useful as a tool for quickly screening 
large numbers of materials for desirable thermodynamic characteristics. In one embodiment, 
data processing system 140 receives images at a rate of 12 to 20 frames per second and reduces 
the data by averaging values for each frame to obtain the equivalent of one averaged frame of 
data per second, thereby increasing the signal to noise ratio of the experimental results. 
[0051] Depending on the process being monitored, different data acquisition rates — for 
example, rates of about 1, 6, 12 or 20 frames per second ~ provide a data stream suitable for 
deriving a reaction profile sufficient to track many chemical reactions, material transformations 
and thermal diffusion transients. Higher data acquisition rates are useful in characterizing 
processes requiring a relatively high degree of resolution. For example, metal compositions can 
be characterized based on the range of temperatures at which melting occurs. For many metal 
compositions, melting begins at one temperature and continues until the material reaches a 
higher temperature. The first temperature, below which the entire sample is in a solid phase, is 
known as the solidus temperature; the second, above which the entire sample is in a liquid phase, 
is known as the liquidus temperature. To identify eutectic compositions — those in which the 
solidus temperature and liquidus temperature are equal — it is useful to screen an array of metals 
covering a range of compositions for those having the lowest liquidus temperature. By rapidly 
heating such an array, it is possible to exceed a composition's solidus temperature without the 
onset of melting due to slow diffusion in the solid state. If heating is sufficiently rapid — for 
example at 20° to 60°C per second — it is possible to reach the liquidus temperature without 
melting, at which point the entire sample quickly melts. Data acquisition at speeds of 1 8 to 20 
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frames per second are sufficient to resolve such rapid thermodynamic changes. 
[0052] The features and advantages of the invention can be appreciated from a description of its 
application to a particular implementation in the field of infrared thermography the 
characterization of thermoelectric materials. 

Thermoelectric Materials 
[0053] Thermoelectric devices are solid state devices that pump heat from one junction to 
another when subjected to an electric current, a phenomenon known as the Peltier effect. 
Thermoelectric materials are characterized by a number of physical parameters including the 
thermal conductivity 6, the electrical conductivity O, the Seebeck coefficient S, the Hall 
coefficient R H , the charge carrier mobility :, the device operating temperature T, the charge 
carrier effective mass m*, and the band gap E g . An estimate of the thermoelectric device 
efficiency for pumping heat relative to the heat lost due to the electrical resistivity is the 
thermoelectric figure of merit defined by: 



S 2 oT 



K 

[0054] FIG. 10 illustrates a model thermoelectric device constructed from a thermoelectric 
material 1000 connected to voltage source 1010 with voltage wires 1020 made of a high 
conductivity metal. Under steady state or adiabatic conditions, the heat pumped by the Peltier 
effect will be equal to the heat carried by the thermal conduction, which leads to the following 
relationship for the thermoelectric figure of merit: 

%T = Q peltier — ^Peltier 
Qjoule Pjoule 

[0055] Qpeitier and Qj 0 uie are the amount of heat transported by the Peltier effect and the amount 
of energy lost to Joule heating, respectively; Pp e itier and P Jou ie are each the corresponding power 
(heat per unit time). The expression for the total power dissipated in the device is given by the 
sum of the thermoelectric (Peltier) and Joule components: 
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v 2 V 

P ~ P Joule + ^ Peltier ~ + ^ ~ 

where the first term is the power dissipated by Joule effects (Pj ou ie) and the second term is the 
contribution from the Peltier component (Pp e itier); R is the electrical resistance of the circuit and 
A is the Peltier coefficient of the thermoelectric material measured relative to the material that 
makes up the voltage contacts. Application of an oscillatory voltage at a reference frequency cd 0 , 
such as V(t) = V 0 cos(cDot), causes the Joule term to oscillate at twice the reference frequency 
(2(o 0 ) due to the voltage being squared, while the Peltier term is linear in voltage and is observed 
at the reference frequency o>o- Measurement of the power dissipated at the junction between the 
voltage contact and the thermoelectric material as a function of time, P(t), followed by a Fourier 
transform to power as a function of frequency, P(co), allows the contribution due to Joule effects 
to be distinguished from (and compared to) the contribution due to Peltier effects. The ratio of 
the two amplitudes P(a>o)/P(2a>o) * s the thermoelectric figure of merit ZT. 
[0056] The present invention provides a fast and efficient screen for the identification of 
thermoelectric materials in a combinatorial library of candidate materials, as well as an analytical 
tool for characterizing such materials by determining the thermoelectric figure of merit. Suitable 
libraries can be created using techniques such as those described in U.S. Patent No. 5,776,359, 
which is incorporated herein by reference in its entirety. 

[0057] As shown in FIG. 1 1, in combinatorial thermoelectric device 1 100, voltage source 1110 
applies a parallel voltage through voltage wires 1 120 to all members 1 130 in a combinatorial 
library 1 140. Library 1 140 is made from a material with low thermal conductance, such as 25 to 
50 micron thick polyimide sheets to minimize heat lost to the substrate holding elements 1 130; 
for the same reason, library 1 140 is maintained in an evacuated environment to minimize heat 
lost to the surroundings. To maintain adiabatic conditions, the voltage applied by voltage source 
1 1 10 is kept as small as possible, with reference frequencies typically being on the order of 0.1 
Hz or less. Voltage wires 1 120 are made of a high conductivity metal such as Ag, Au, Cr, Ta or 
Cu to maintain good electrical contact and because their Peltier coefficients are low relative to 
those found in semiconducting materials. 

[0058] As discussed more generally above, data reduction module 150 obtains a series of 
images of thermoelectric device 1 100 from infrared camera 100. Data reduction module 150 
generates reduced data values as described above, and provides those values to data analysis 
module 160. 
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[0059] With these reduced data values, data analysis module 160 calculates the intensity as a 
function of frequency for each region of interest using fast Fourier transform. As shown in 
FIGS. 12A and 12B, the user can view the data graphically, in either the time domain 1200 or the 
frequency domain 1250. Data analysis module 160 uses the frequency data to calculate the 
thermoelectric figure of merit for each region of interest as described above. The figure of merit 
(1220, 1270) is displayed in thumbnail graphs 1210 and 1260. The user can also view a three 
dimensional histogram displaying the figure of merit for each region of interest, as described 
above. 

[0060] Chemical Transformations 
[0061] Data processing system 140 can also be used to monitor the progress of chemical 
reactions or transformations. In many embodiments, the reaction is a catalytic reaction and 
catalyst activity is being determined, which is the example described below. Referring to FIG. 
13, in this application an infrared camera 100 monitors emittance associated with heat evolution 
or absorption or with a compound or chemical's disappearance from or appearance in a 
combinatorial library 1310 under various external conditions such as temperature and gas flow. 
For example, if a solid catalyst library and its surrounding support in a two-dimensional library 
are exposed to a reactant, a measurable heating of the surroundings may occur depending on the 
activity of the chemical process. Such libraries can be generated using techniques such as those 
disclosed in U.S. Applications Serial No. 08/898,715, filed July 22, 1997, or Serial 
No. 09/227,558, filed January 8, 1999, which are both incorporated herein by reference in their 
entirety. In the case of a catalyst, the activity of the catalyst correlates to the energy released or 
absorbed as heat during the chemical reaction between the catalyst and the exchange gas. In a 
combinatorial library 1310, members 1320 are nearly identical in thermal mass, so measurements 
of the heat evolved by one element in the library relative to others within the library can be used 
to characterize the chemical processes induced by these materials. 
[0062] Figure 14 illustrates a two-dimensional library 1400 of materials useful in this 
embodiment. The individual library members are reaction wells 1410 in a substrate 1420. 
Substrate 1420 is placed within a sealed reaction chamber which is pressurized with the relevant 
gas. Windows 1430 and 1440 are made of an infrared transparent medium (such as sapphire, 
antireflection coated silicon, BaF 2 , CaF 2 or NaCl) capable of maintaining the pressurized gas 
inside the chamber. Since windows 1430 and 1440 are transparent to infrared radiation, thermal 
imaging techniques can be used to monitor the heat of reaction of each library element under 
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various external conditions. 

[0063] Measuring the heat generated by a catalytic transformation is a powerful technique for 
rapidly screening catalyst efficacy. For condensed phase reactions, there is direct thermal 
contact between the catalyst, products, and solvent. Thus, the library can be directly imaged and 
the emissivity of the samples will be roughly that of the pure solvent, allowing direct comparison 
of the members in a library. In cases where emissivity differences exist between materials in the 
library, direct comparison of library members is more complicated, but the experimental results 
are still useful in screening for raw catalyst activity. Differences in emissivity may also be dealt 
with by imaging the library through a material (such as graphite). This technique is particularly 
useful for screening gas phase reactions, where the catalyst is mounted on a support and large 
emissivity differences are common. In this case, the reaction can be monitored from the 
backside of the catalyst support. 

[0064] In a typical experiment, a library is loaded into a sample chamber with an IR transparent 
window and the entire system is allowed to come to thermal equilibrium. The sample is 
monitored with camera 100 to establish a background reading, and the chamber is pressurized 
with a reactant gas. The sample is monitored by camera 100 over the course of the catalytic 
reaction, for example, for about one hour. The resultant data is analyzed as will be described 
below. The change in emittance is directly proportional to the catalytic activity of the library 
member. 

[0065] As discussed more generally above, data reduction module 150 obtains a series of 
images of library 1400 from infrared camera 100. Data reduction module 150 generates reduced 
data values as described above, and furnishes those values to data analysis module 160. 
[0066] With these reduced data values, data analysis module 160 calculates the integral of 
intensity (proportional to the total conversion of starting material). As described above, the user 
can view the data in the form of an array of graphs 800 of intensity versus time and integral of 
intensity versus time. Each thumbnail graph also depicts two figures of merit for each region of 
interest — the total integrated intensity and the maximum intensity for the element. The user can 
also view a three dimensional histogram displaying either figure of merit for each region of 
interest. 

Metals 

[0067] Data processing system can also be used to characterize melting points of metals or other 
materials (e.g., composite materials) using infrared imaging. In this application, an infrared 
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camera 100 is used to monitor a library whose members include a series of metals or other 
materials. Suitable libraries can be generated using techniques such as those described in U.S. 
Patent No. 5,776,359, which is incorporated herein by reference in its entirety. Camera 100 
measures intensities that are a function of temperature and emissivity of each member. Changes 
in the structure and bonding of a chemical composition during a transition from one 
thermodynamic phase to another result in a change in the composition's emissivity. During a 
phase transition the temperature of a library member may change or the rate of temperature 
change may increase or decrease. The thermodynamics of a given material's phase transition 
can be characterized by comparing the emissivity changes over a given temperature range with 
that of a standard material having a constant emissivity in the relevant temperature range. 
[0068] The infrared camera 100 monitors the intensity from every library member in parallel 
and compares it to the intensity of a known standard material within the field of view of the 
camera (which may be the substrate holding the library) and subjected to the same physical 
conditions as the library members. In this way, complicated phase relationships are measured for 
large libraries of materials by heating or cooling the library and deriving changes in the 
emissivity. 

[0069] Referring to FIG. 15, as discussed more generally above, in this embodiment camera 100 
records infrared images of library 1500, including library members 1510, as the temperature is 
varied over a range of interest. Data reduction module 150 obtains a series of images of library 
1 500 and generates reduced data values as described above. Data reduction module 150 
furnishes those values to data analysis module 160. 

[0070] Using these reduced data values, data analysis module 160 calculates the relative 
emissivity of each library member as compared to the standard. After smoothing the data, data 
analysis module 160 calculates the derivative of the ratio of library member emissivity to 
standard. The maximum rate of change occurs at the material's melting point which here is the 
figure of merit. As described above, the user can view the experimental results as an array of 
thumbnail graphs 800 depicting a plot of intensity versus time for a given library member, 
intensity versus time for the standard sample (which may be the substrate) and a plot that is a 
ratio of the first two plots. The user can also view a reduced plot of melting point as a function 
of library member. The figure of merit — the melting point calculated as described above — is 
also displayed. As described above, the user can also view the data as a three dimensional 
histogram displaying either figure of merit for each library element. 
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[0071] The invention can be implemented in digital electronic circuitry, or in computer 
hardware, firmware, software, or in combinations of them. Apparatus of the invention can be 
implemented in a computer program product tangibly embodied in a machine-readable storage 
device for execution by a programmable processor; and method steps of the invention can be 
performed by a programmable processor executing a program of instructions to perform 
functions of the invention by operating on input data and generating output. The invention can 
be implemented advantageously in one or more computer programs that are executable on a 
programmable system including at least one programmable processor coupled to receive data and 
instructions from, and to transmit data and instructions to, a data storage system, at least one 
input device, and at least one output device. Each computer program can be implemented in a 
high-level procedural or object-oriented programming language, or in assembly or machine 
language if desired; and in any case, the language can be a compiled or interpreted language. 
Suitable processors include, by way of example, both general and special purpose 
microprocessors. Generally, a processor will receive instructions and data from a read-only 
memory and/or a random access memory. Storage devices suitable for tangibly embodying 
computer program instructions and data include all forms of non- volatile memory, including by 
way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory 
devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; 
and CD-ROM disks. Data can also be temporarily stored in volatile memory. Any of the 
foregoing can be supplemented by, or incorporated in, ASICs (application-specific integrated 
circuits). 

[0072] To provide for interaction with a user, the invention can be implemented on a computer 
system having a display device such as a monitor or LCD screen for displaying information to 
the user and a keyboard and a pointing device such as a mouse or a trackball by which the user 
can provide input to the computer system. The computer system can be programmed to provide 
a graphical user interface through which computer programs interact with users. 
[0073] An example of one such type of computer is shown in FIG. 16, which shows a block 
diagram of a programmable processing system (system) 1600 suitable for implementing or 
performing the apparatus or methods of the invention. The system 1600 includes a processor 
1610, a random access memory (RAM) 1620, a program memory 1630 (for example, a writable 
read-only memory (ROM) such as a flash ROM), a hard drive controller 1640, and an 
input/output (I/O) controller 1650 coupled by a processor (CPU) bus 1660. The system 1600 can 
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be preprogrammed, in ROM, for example, or it can be programmed (and reprogrammed) by 
loading a program from another source (for example, from a floppy disk, a CD-ROM, or another 
computer). 

[0074] The hard drive controller 1640 is coupled to a hard disk 1645 suitable for storing 
executable computer programs, including programs embodying the present invention, and data 
including the images, masks, reduced data values and calculated results used in and generated by 
the invention. 

[0075] The I/O controller 1650 is coupled by means of an I/O bus 1670 to an I/O interface 
1680. The I/O interface 1680 receives and transmits data (e.g., stills, pictures, movies, and 
animations for importing into a composition) in analog or digital form over communication links 
such as a serial link, local area network, wireless link, and parallel link. 

[0076] Also coupled to the I/O bus 1670 is a display 1690 and a keyboard 1695. Alternatively, 
separate connections (separate buses) can be used for the I/O interface 1 670, display 1690 and 
keyboard 1695. 

[0077] The invention has been described in terms of particular embodiments. Other 
embodiments are within the scope of the following claims. For example, the invention has been 
described as being implemented in a number of different embodiments, each intended to 
monitor, analyze and display data for a different experiment. Alternatively, a number of 
different applications can be implemented in a single system, for example, where data processing 
system 140 includes multiple versions of data analysis module 140, each designed to analyze and 
display data from a different type of experiment. Also, data structures other than the ones 
mentioned above can be used in storing and processing data. For example, mask and image 
information can be encapsulated in objects and stored in an object oriented database. In addition, 
the steps of the invention can be performed in a different order and still achieve desirable results. 
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